Small and thin handheld portable devices are preferred by many users. In relation to picture taking, this presents a need for thin imaging systems. Camera size, particularly in terms of thickness, is related to the size of the image plane. A normal lens focal length for a camera is generally accepted to be equal to the diagonal of an electronic imager or a frame of photographic film. By definition, meeting this criterion produces “correct” perspective or unit angular magnification. With a short focal length lens, the film or imager has to have a small diagonal. This leads to large print magnification for film systems or small pixels that tend to be noisy for electronic systems.
An array of lenses can be used with an imager to capture an array of images of a scene. “Opto-Electronic Integrated Information System”, by Tanida, J. et al., IEICE Trans. Electron., Vol. E84-C, No. 12, December 2001, pages 1778-1784, discloses a system having an array of lenses over an imager and a separator that prevents cross talk between lenses. Such a system can include a color filter array, as disclosed in US 2003/0111593. The normal lens focal length for this type of system is the diagonal of the small section of the imager that the respective lens covers. Because the focal length of the individual lenses is shorter than it would be for the whole imager, this provides a thin optical system. The multiple images of the same scene are each captured at a relatively low resolution. The images can be added together to reduce noise and/or increase resolution. The pixels under each lens can be offset by a small part of a pixel pitch so that each image will have the same pixel pitch with a small offset. The separate images can be used to fill in a higher sampling frequency grid than the grid under one lens. Alternatively, if the images are identical and aligned to the pixel pitch, then multiple images can be averaged pixel by pixel to produce a lower noise image.
These systems have not addressed parallax issues for objects that are close to the camera. For example, a system based on a current megapixel sensor with approximately 4 micron pixels requires a 3 mm focal length lens. In this case the foreground will be shifted relative to the background about 2 pixels in the captured images. The shift leads to a loss of resolution and, depending on the system design, color fringes in the image. Therefore, there exists a need to address parallax issues for images captured by thin imaging systems at distances less than infinity.
Autocorrelation of images of the same scene from different sources is well known. U.S. Pat. No. 5,835,639 uses autocorrelation in detecting rotation and magnification of an image bearing an embedded marker. US 2002/0061131 discloses the use of correlation and warping in the preparation of depth images. U.S. Pat. No. 5,453,840 discloses a procedure for aligning two sensors using cross correlation. U.S. Pat. No. 5,530,514 discloses a system for autofocusing by deriving parallax range using autocorrelation of signals from two linear sensor arrays.
It would thus be desirable to provide methods and apparatus, in which parallax is corrected in producing final images from a plurality of images captured by sections of a capture unit, each section having one of an array of lenses.